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ABSTRACT 

A  spectral  imager  constructs  a  three  dimensional  (two  spatial  and  one  spectral)  image  from  a  series 
of  two  dimensional  images.  This  paper  discusses  a  technique  for  spectral  imaging  that  multiplexes  the 
spatial  and  spectral  information  on  a  staring  focal  plane,  then  demultiplexes  the  resulting  imagery  to 
obtain  the  spectral  image.  This  approach  has  high  optical  throughput  and  is  robust  to  focal  plane 
array  (FPA)  nonuniformities.  This  presentation  is  the  tliird  in  a  series  to  the  IRIS  passive  sensors 
community.  Previously  we  introduced  the  hardware  and  discussed  the  reconstruction  algorithm  in 
detail.  Here  we  review  the  hardware  and  present  experimental  results. 


1.  INTRODUCTION 

Often  objects  of  similar  intensity  have  different  spectral  signatures  which  can  be  used  to  distinguish 
between  them.  This  work  is  motivated  by  our  desire  to  perform  spectral  discrimination  in  the  mid¬ 
wave  infrared.  From  previous  experience  we  know  that  infrared  scenes  consist  of  low  contrast  details 
on  a  large  background  pedestal,  and  that  focal  plane  array  (FPA)  nonuniformities  and  low  optical 
throughput  often  hmit  the  performance  of  infrared  spectral  imagers  [1].  Our  goal  is  to  address  these 
potential  performance  hmiting  issues  by  incorporating  both  a  throughput  and  a  multiplex  advantage 
into  a  spectral  imager,  and  at  the  same  time  address  the  effect  of  FPA  nonuniformity  on  spectral 
imaging. 

Typical  state-of-the-art  spectral  imagers  image  the  spectrum  of  a  sht  onto  a  2-dimensional  focal  plane 
array;  the  sht  is  scanned  over  the  object  to  create  a  3-dimensional  spectral  image  [2].  Since  only  a  sht 
is  imaged  at  a  time,  the  out-of-sht  photons  are  rejected,  hence  these  spectral  imagers  are  inherently 
inefficient.  Further,  pixel-to-pixel  nonuniformities  on  the  FPA  can  corrupt  the  spectral  imagery  and 
reduce  the  ultimate  performance. 

The  spectral  imager  we  study  is  similar  to  previously  reported  chromotomographic  devices  [3,  4,  5,  6,  7], 
ah  of  which  image  a  dispersed  view  of  the  entire  scene  over  ah  of  the  spectral  bands,  then  computa- 
tionahy  reconstruct  the  chromatic  image.  In  chromotomography  the  color  imagery  is  considered  to  be 
pseudo-three  dimensional.  The  3D  spectral  image  is  obtained  using  tomographic  imaging  techniques 
similar  to  those  used  in  medical  imaging.  While  chromotomography  is  computationahy  demanding,  it 
has  the  advantage  that  a  sht  held  stop  is  not  reqiured.  We  anticipate  the  spectral  imager  performance 
wih  rehect  the  larger  optical  throughput  afforded  by  a  larger  held  stop.  The  actual  performance  and 
potential  multiplex  advantage  (or  disadvantage),  depend  on  how  weh  the  reconstruction  algorithms 
perform. 

Our  technique  differs  from  other  chromotomographic  approaches  in  several  ways:  we  utihze  a  direct 
vision  prism  to  disperse  the  hght;  we  have  derived  a  reconstruction  method  that  is  uniquely  suited  to 
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Figure  1:  A  schematic  representation  of  the  spectral  imager  and  its  operation.  On  the  left  the  direct 
vision  prism  is  shown  spreading  red,  green  and  blue  light  across  the  FPA.  On  the  right  the  effect  of 
the  prism  rotation  on  the  overlapping  red,  green  and  blue  images  is  illustrated.  The  circle  and  arrow 
represent  the  prism  orientation. 


our  hardware  configuration  [8];  we  have  developed  some  new  ideas  for  dealing  with  some  of  the  hmi- 
tations  of  chromotomography,  and  have  applied  principal  component  analysis  to  the  characterization 
of  our  experimental  data.  Here  we  describe  the  hardware,  the  reconstruction  technique,  and  the  limi¬ 
tations  of  our  approach.  We  demonstrate  infrared  spectral  imaging  and  demonstrate  infrared  spectral 
discrimination.  We  will  also  demonstrate  that  the  imager  is  robust  to  the  FPA  nonuniformities  that 
often  limit  infrared  imager  performance. 


2.  HARDWARE  CONFIGURATION  AND  RECONSTRUCTION  TECHNIQUE 

Figure  1  is  a  schematic  representation  of  our  approach.  It  consists  of  a  telescope,  a  direct  vision 
prism  and  a  camera.  A  direct  vision  prism  consists  of  two  prisms  that  are  arranged  such  that  one 
wavelength  passes  through  undeviated  while  other  wavelengths  are  deviated  along  a  hne.  The  direct 
vision  prism  is  mounted  in  a  bearing  so  that  it  can  be  rotated  around  the  optical  axis.  As  the  prism 
rotates  the  spectral  features  in  the  image  will  trace  out  circles  where  the  radius  of  the  circles  are 
wavelength  dependent.  To  obtain  a  spectral  image,  a  series  of  frames  of  camera  data  are  obtained 
with  the  prism  in  different  angular  orientations,  then  a  computer  is  used  to  extract  the  spectral  image 
from  the  measured  data. 

If  in  Fig.  1  the  held  stop  were  replaced  by  a  sht,  the  prism  mount  were  hxed,  and  a  mechanism  for 
hnearly  scanning  the  scene  added,  then  the  modihed  system  would  be  a  standard  scanned-sht  spectral 
imager.  In  both  cases  the  throughput  is  determined  by  the  and  the  held  stop  area.  Since  the 
//^’s  are  the  same  and  the  held  stop  area  is  much  larger  for  the  angularly  multiplexed  conhguration, 
it  has  higher  throughput. 

In  its  current  conhguration  the  spectral  imager  gathers  one  frame  of  data  from  each  of  80  prism 
orientations  uniformly  spaced  over  27r.  The  //4  InSb  camera  operates  at  60  fps  with  a  2ms  integration 
time.  Complete  sequences  of  data  are  gathered  in  1.33s  with  a  composite  integration  of  160ms.  Future 
conhgurations  could  increase  the  frame  rate  to  improve  the  dwell  efficiency  of  the  instrument.  Frame 
rates  of  up  to  500  fps  could  be  achieved  without  signihcant  degradation  of  the  imager  performance.  At 


these  high  frame  rates,  complete  spectral  Images  could  be  obtained  at  near  real-time  rates  (complete 
spectral  images  in  less  than  0.2s). 

The  algorithm  used  to  reconstruct  the  spectral  imagery  from  the  raw  data  is  combination  of  a  direct 
inverse  and  an  iterative  algorithm[9].  The  advantage  of  the  combined  approach  is  that  the  known 
characteristics  of  spectral  imagery  can  be  utilized  to  overcome  some  of  the  potential  hmitations  of 
the  tomographic  approach.  Currently  the  reconstruction  algorithm  is  implemented  on  a  two  processor 
Pentium  Pro  computer.  Complete  reconstructions  are  currently  obtained  in  less  than  three  minutes; 
however  a  signihcant  reduction  in  the  run-time  could  be  achieved  by  sacrihcing  Ilexibihty. 


3.  EXAMPLE  IMAGERY 

In  order  to  evaluate  the  capabihties  of  the  instrument,  we  imaged  a  target  of  opportunity  (a  building) 
from  a  laboratory  window.  The  data  was  obtained  on  January  12,  1998  at  9:30  in  the  morning.  In 
order  to  minimize  the  effects  of  FPA  nonuniformities  and  stray  reflections  from  the  rotating  prism, 
an  additional  sequence  of  data  was  recorded  with  the  entrance  pupil  blocked.  The  blocked  sequence 
was  used  to  perform  a  one-point  nonuniformity  correction  on  a  frame-by-frame  basis.  This  type  of 
nonuniformity  correction  reduces  the  ultimate  signal  to  noise  ratio  by  a  factor  of  -s/2.  Development  of 
an  optimal  nonuniformity  correction  technique  remains  for  future  work. 

Examples  of  the  dispersed  imagery  is  shown  in  Fig.  2.  The  angular  orientation  of  the  prism  is 
reflected  in  the  imagery  by  a  anisotropic  blurring  in  the  direction  of  prism  dispersion.  Sharp  edges  in 
the  scene  are  sharply  Imaged  only  when  the  prism  orientation  matches  that  of  the  edge.  Since  edges 
are  sharply  focused  for  some  of  the  prism  orientations,  the  information  about  the  edges  is  not  lost  in 
the  same  sense  that  it  is  lost  when  the  imagery  is  defocused.  (This  differentiates  tomography  from 
image  enhancement.) 

Also  evident  in  the  raw  data  is  the  effect  of  atmospheric  absorption  by  C02-  The  effect  is  clearly 
visible  in  the  90  deg  orientation  image  as  a  horizontal  dark  band  separating  two  bright  regions. 

Figure  3  illustrates  6  of  the  64  spectral  reconstructions.  Though  there  are  64  samples,  only  33  of 
them  are  statistically  Independent  (due  to  filtering  in  the  reconstruction  algorithm[8]);  therefore  the 
spectral  resolution  is  approximately  0.06  jim  or  40  cm  Spectral  profiles  of  a  selection  of  pixels  in 


Figure  2:  Three  images  of  the  Eighty  frame  sequence.  The  prism  orientations  were  0,  45  and  90  deg. 


Figure  3:  Spectral  reconstructions.  From  right  to  left  across  the  top  row  3.52,  3.95,  and  4.14/nm,  and 
4.25,  4.38  and  4.62/im  across  the  bottom  row. 
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Figure  4:  Spectral  profiles  of  four  pixels  in  the  image.  The  profile  labeled  ’Solar’  is  that  of  an  aluminum 
vent  on  the  roof.  The  profile  labeled  ’thermal’  is  that  of  an  air  conditioning  unit  also  on  the  roof.  The 
two  mixed  profiles  are  from  different  sections  of  wall,  one  brick  and  the  other  painted  metal. 


the  image  are  illustrated  iu  Fig.  4.  The  spectral  profiles  are  taken  from  sequences  of  data  after  the 
mean  of  each  frame  has  been  subtracted  on  a  frame-by-frame  basis.  The  mean  removal  highhghts  the 
spectral  features  at  the  expense  of  the  monotonic  upward  trend  in  intensity  from  the  3  to  5  fim. 

Solar  radiation  dominates  the  3.0  /rm  side  of  the  band,  and  thermal  emission  dominates  the  5.0  fim 
side.  In  general  the  cross-over  depends  on  the  time  of  day,  the  weather  and  the  ambient  temperature. 
For  our  scene  the  cross-over  occurs  at  3.95  /rm;  the  effect  of  which  can  be  seen  in  the  reversal  of 
contrast  of  the  windows  on  the  building  in  the  sequence  of  images.  The  vents  on  the  roof  are  visible 
at  3.52  jim  but  not  at  4.62  /rm,  while  the  converse  is  true  for  the  air  conditioner. 

The  strong  effect  of  atmospheric  absorption  over  the  4.2-4.5  jim  band  is  observed  as  a  dramatic 
reduction  in  contrast  in  the  imagery,  and  as  zero  intensity  on  the  spectral  profiles  (where  the  spatial 
mean  has  been  removed).  The  spectral  profiles  also  exhibit  the  broadening  of  the  absorption  band 
with  range.  The  thermal  signatures  are  emitted  at  the  building,  and  must  pass  through  only  the 
atmosphere  between  the  building  and  the  camera.  The  solar  profiles,  being  reflected,  must  pass 
through  the  atmosphere  between  the  sun  and  the  building,  then  pass  through  the  atmosphere  between 
the  building  and  the  camera.  The  observed  profiles  exhibit  the  effects  of  the  different  path  lengths. 
(Another  example  for  very  short  paths  is  illustrated  in  Fig.  7.) 

An  artifact  of  the  instrument  that  is  nearly  coincident  with  the  CO2  absorption  band  is  the  effect  of 
the  FPA  nonuniformities.  The  direct  vision  prism  has  been  constructed  such  that  the  light  at  4.38  /rm 
passes  through  undeviated.  Consequently,  as  the  prism  rotates,  the  portion  of  the  spectral  image  near 
4.38  iim  remains  fixed  on  the  focal  plane.  As  far  as  the  inversion  algorithm  is  concerned,  there  is  no 
way  to  distinguish  between  the  focal-plane-array  nonuniformities  and  the  4.38  /rm  light.  Therefore  the 
effect  of  the  nonuniformities  is  mapped  into  the  4.38  pm  spectral  band.  If  the  undeviated  wavelength 
is  designed  to  coincide  with  a  strong  absorption  band,  then  the  effect  of  the  focal  plane  nonuniformities 
on  the  spectral  discrimination  function  can  be  minimized. 

The  spectral  profiles  shown  in  Fig.  4  provide  a  way  of  verifying  the  spectral  resolution.  The  spectral 
feature  present  as  a  shoulder  in  the  thermal  profile  and  as  a  dip  in  one  of  the  mixed  profiles  at  4.9 
jim  is  due  to  the  atmosphere.  MODTRAN  atmospheric  absorption  simulations  indicate  that  if  the 
spectral  resolution  were  much  poorer  than  20  cm  ^  the  feature  would  not  be  visible.  Our  Calculations 
indicate  that  at  4.9  pm  the  spectral  resolution  should  be  24  cm  We  conclude  that  the  spectral 
resolution  is  close  to  the  design  resolution. 

By  design  the  spectral  resolution  near  3  pm  is  110  cm  The  variation  in  spectral  resolution  as  a 
function  of  wavelength  is  due  to  the  nonlinear  dispersion  of  the  direct  vision  prism.  On  average,  the 
spectral  resolution  across  the  3-5  pm  band  is  40  cm 

4.  PRINCIPAL  COMPONENT  ANALYSIS  AND  END-MEMBERS 


A  widely  used  tool  for  finding  correlations  and  locating  anomalies  in  a  data  set  is  Principal  Component 
Analysis  (PCA)[10].  The  goal  of  PCA  as  applied  to  spectral  imaging  is  to  uncover  band-to-band 
correlations  among  the  spectral  images,  then  identify  a  set  of  spectra  and  corresponding  images  that 
have  no  mutual  correlation,  but  can  stiU  fully  represent  the  spectral  imagery.  PCA  is  an  effective 
tool  in  hyperspectral  image  processing  because  spectral  images  exhibit  a  large  degree  of  correlation. 
Once  the  strongly  correlated  features  are  grouped  together,  the  anomalous  features  become  more 


Figure  5:  Two  views  of  the  hyperspectral  image  scatter  plot.  The  left  image  illustrates  the  hrst 
eigeuimage  intensity  versus  the  second.  The  right  image  illustrates  the  second  versus  fourth.  The 
third  eigenimage  represents  focal  plane  array  nonuniformities. 


Figure  6:  Images  of  three  of  the  end-members  of  the  hyperspectral  imagery.  From  left  to  right  the 
images  correspond  to  the  ’Solar’,  ’Thermal’,  and  ’Painted  Metal’  end-members. 


pronounced. 

The  process  of  PCA  involves  identifying  an  orthoiiormal  basis  of  eigenspectra.  Each  of  the  eigenspectra 
has  a  corresponding  eigenimage.  The  intensity  of  a  pixel  in  a  given  eigenimage  estabhshes  the  intensity 
of  the  corresponding  eigenspectrum  for  that  pixel.  Figure  5  is  a  scatter  plot  where  each  pixel  is 
represented  by  a  bright  point,  and  the  coordinates  of  the  point  are  estabhshed  by  two  eigenimage 
intensities.  Pixels  with  similar  spectral  signatures  form  clusters  on  the  scatter  plot. 

The  three  bright  clusters  in  the  left  image  of  Fig.  5  are  due  to  the  held  stop,  the  ground  and  the  sky. 
One  can  imagine  the  three  bright  clusters  as  one  vertex  of  a  larger  triangle  that  envelops  aU  of  the  points 
in  the  scatter  plot.  The  top  vertex  of  the  triangle  represents  those  pixels  that  have  a  solar  dominated 
spectrum,  the  bottom  vertex  represents  those  pixels  that  have  a  thermal  dominated  spectrum,  and 
the  left  vertex,  with  the  three  large  clusters,  represents  the  reference  from  which  the  other  points  are 
measured.  The  vertices  of  this  triangle  represent  the  end-members  of  our  hyperspectral  image.  They 
are  the  purest  examples  of  spectral  behavior  that  is  anomalous  from  the  background  pixels.  Spectral 
prohles  of  the  two  pixels  closest  to  the  vertices  of  the  triangle  are  labeled  ’Solar’  and  ’Thermal’  in 


Fig.  4.  The  right  image  in  Fig.  5  exposes  an  additional  end-member,  that  due  is  to  the  painted  metal 
structure  on  the  roof  of  the  building.  Images  of  these  three  end-members  are  shown  in  Fig.  6. 


5.  PERFORMANCE  CHARACTERIZATION 

Consider  the  schematic  representation  of  our  instrument  in  Fig.  1.  If  the  square  held  stop  were 
replaced  by  a  sht  with  the  thin  dimension  aligned  in  the  direction  of  prism  dispersion,  the  hardware 
would  be  conhgured  as  a  scanned  sht  instrument.  Since  the  optics,  camera  and  electronics  are  identical, 
a  comparison  of  the  performance  in  the  two  conhgurations  will  provide  a  measure  of  the  tomographic 
spectral  imager  performance  relative  to  that  of  a  scanned  sht  instrument. 

Such  a  comparison  is  made  in  Fig.  7.  A  uniform  target  was  measured  in  both  conhgurations.  The 
data  in  the  scanned  sht  conhguration  was  obtained  from  one  row  of  pixels.  The  cahbration  is  identical 
in  each  case  and  was  obtained  by  averaging  over  50  rows  and  over  64  frames  of  the  scanned  sht  data. 
A  comparison  of  the  two  plots  in  Fig.  7  indicates  that  the  performance  of  the  tomographic  approach 
is  superior  to  that  of  the  scanned  sht  by  nearly  a  factor  of  3. 

If  the  sht  in  this  experiment  were  cold,  then  the  performance  of  the  scanned  sht  conhguration  may 
improve  relative  to  the  tomographic  conhguration.  A  cold  sht  would  have  ehminated  the  thermal 
emission  from  the  sht,  which  would  lead  to  a  reduction  in  the  noise  due  to  the  background  hux.  In 
general,  we  conclude  that  the  performance  of  the  tomographic  approach  wih  be  superior  to  that  of 
a  scanned  sht  when  the  noise  is  not  dominated  by  the  shot  noise  of  the  signal  photo-electrons.  In 
addition  to  a  warm  sht,  short  data  acquisition  time,  low  hux  levels  and  limited  optical  aperture  can 
keep  an  optical  system  from  being  signal  shot  noise  hmited.  If  the  performance  were  signal  shot 
noise  hmited,  the  performance  of  tomograpliic  technique  would  be  approximately  a  factor  of  3  poorer 
than  that  of  the  scanned  sht  for  the  current  reconstruction  algorithm.  We  are  hopeful  that  future 
improvements  wih  lead  to  an  even  more  favorable  comparison. 

Figure  8  ihustrates  the  Modulation  Transfer  Function  of  both  the  infrared  camera  and  the  spectrometer 
as  measured  using  the  modihed  knife-edge  technique  [11].  (The  infrared  camera  was  measured  with 
the  re-imaging  relay,  and  without  the  prism.)  The  comparison  shows  that  MTF  is  unaffected  by  the 
tomographic  reconstruction  for  spatial  frequencies  below  1/2  Nyquist.  However,  above  1/2  Nyquist 
some  degree  of  degradation  is  present. 


6.  CONCLUSIONS 


The  tomographic  approach  to  spectral  imaging  provides  high-lidehty  data  with  simple  hardware.  There 
is  no  need  for  a  sht  in  the  tomographic  approach  which  significantly  enhances  the  optical  throughput 
and  removes  any  potential  requirement  for  coohng  the  sht  or  any  of  the  optics. 

The  average  spectral  resolution  across  the  3-5  iim  band  is  approximately  40  cm  Near  5  fim  the 
spectral  resolution  is  comparable  to  the  20  cm  ^  design  hmit  of  the  LOTR.AN  atmospheric  model. 

The  spatial  resolution  of  the  reconstructed  spectral  imagery  is  only  marginahy  degraded  relative  to 
the  pan-chromatic  imagery  taken  with  the  same  hardware.  The  signal  to  noise  ratio  of  the  resulting 


Figure  7:  Comparison  of  spectral  profiles  obtained  with  the  tomographic  configuration  (left)  and  in  a 
scanned  sht  configuration  (right). 


Figure  8:  Measured  MTF  of  the  infrared  camera  and  the  spectral  imager. 


imagery  is  superior  to  that  of  read-uoise  limited  scanned  slit  configurations  that  utifize  the  same 
hardware. 
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